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Abstract 

Given a string of characters, the Burrows- Wheeler Transform re- 
arranges the characters in it so as to produce another string of the 
same length which is more amenable to compression techniques such 
as move to front, run-length encoding, and entropy encoders. We 
present a variant of the transform which gives rise to similar or better 
compression value, but, unlike the original, the transform we present is 
bijective, in that the inverse transformation exists for all strings. Our 
experiments indicate that using our variant of the transform gives rise 
to better compression ratio than the original Burrows- Wheeler trans- 
form. We also show that both the transform and its inverse can be 
computed in linear time and consuming linear storage. 

1 Introduction 

Informally, the famous Burrows- Wheeler Transform (BWT) [?] can be de- 
scribed as follows. 

Given a string a of length n, generate the n cyclic rotations of a, 
and then sort these. By picking the last character of these sorted 
strings we obtain a string BWT(a), the Burrows- Wheeler trans- 
form of a. 
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BWT has become very popular for text compression application, because of 
two important properties. First, if a is textual, then BWT (a) tends to have 
long runs of identical characters, which makes BWT (a) more amenable to 
compression techniques such as run-length-encoding and move-to- front [?]. 
Consider for example an innocent phrase such as "now is the time for 
the truly nice people to come to the party" , in which there are no 
runs of consecutive identical characters. In contrast, the Burrows- Wheeler 
transform of this phrase becomes 

oewyeeosreeeepi u mhchlmhp u tttnt u puio u yttcef n uu ooati uuuuuuu rrolt 

in which 24 characters participate in eight such runs, the longest of which 
comprises seven repetitions of the space character. To see how these seven 
spaces are brought together, note that the phrase has seven words beginning 
with "t", and hence seven of its rotations begin with "£". When sorted, 
these seven rotations become neighbors, and since the last character in these 
seven is space, seven such spaces occur consecutively in the transform. 

The second important property of the transform is the (rather surprising) 
fact that it is invertible in the following sense: Given BWT(a), it is possible 
to efficiently generate all rotations of a. 

The main issue with the inversion that concerns us here is that, by defi- 
nition, BWT(a) = BWT(q') for every a' which is a rotation of a. Therefore, 
with the absence of additional information, regenerating a given BWT(a) is 
impossible. 

To make decompression possible, compression algorithms must therefore 
store not only the compressed representation of BWT(a), but also the ro- 
tation index, an integer i, < i < n, of a in the sorted list of its cyclic 
rotations. A less "pure" alternative is to append a newly introduced end- 
of-string character □ to a, and then compute BWT(aD). String a is then 
chosen as the rotation in which □ comes last. 

This issue, together with a simple counting argument, shows that BWT _1 (-), 
the inverse Burrows- Wheeler transform, cannot be defined for all strings. In 
fact, if rj is a string of length n selected at random, then BWT" 1 (77) is defined 
with probability 1/n. 

A natural question is whether there exists a similar transform which is 
truly invertible, in the sense that the transformed string uniquely identifies 
the original. In this paper, we answer this question affirmatively by describing 
a bijective, string sorting transform S(-), which is similar to the Burrows- 
Wheeler transform in that it tends to bring identical characters together. In 
fact, in many cases, the output is quite similar to that of the BWT transform. 
For example, applying S to the above phrase yields a string which is different 
in only six locations: 
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yoeyeeosreeeepi u mhchlmhp u tttnt u puio u wttcef n uu ooati uuuuuuu rrotl 

Moreover, the S transform features in this case the same number of runs of 
identical characters and the same number of identical characters participating 
in these runs. Our experimental results indicate that compression revolving 
around S tends to perform (slightly better) than BWT based compression. 

Consider for example, Table [1] which compares the performance of the S 
transform with that of the BWT transform when used for compression the 
famous Calgary corpus, a collection of 18 files which serves as the de-facto 
standard for benchmarking compression algorithms. In preforming the mea- 
surements, we first used Nelson's reference implementation [?], which carries 
out compression in five steps: (i) initial run-length-encoding, (ii) Burrow- 
Wheeler transformation (Hi) move-to- front (iv) yet another run-length-encoding 
(v) arithmetical encoding [?]. We then repeated the same steps, substitut- 
ing 5-transform for BWT in step (ii). (In this particular experiment, we 
relied on the bit- rather than byte- representation of the data. The results 
for byte based compression are similar [?]). 

As can be seen in the table, using the iS-transform, improves compression 
for all files except for Geo. The gain in compression ratio is about 1%. 
Note this gain is much greater than what can be attributed to the saving 
due due bijectivity, that is, the elimination of the end-of-string character, 
or the rotation index: Even in the ProgL file, in which the relative gain 
of the iS-based compression is the smallest, the size saving is of almost 200 
bytes. 

Other than better compression, S offers several other advantages over BWT. 
First, there is no need to store an end-of-string marker, nor the rotation 
index, in applying the Burrows- Wheeler transform in loseless compression 
algorithms. This advantage is prominent especially if the transform is used 
for very short texts, e.g., in transforming separately each line in a text file, 
or each field in database. Second, since the algorithm is bijective, it is more 
adequate for application in which the compressed data is encrypted — a non- 
bijective transform necessarily reveals information to the attacker. Finally, 
some may appreciate the elegance in bijectiveness and in the details of the 
definition of the transform. 

For the impatient reader, an informal (and imprecise) description of the S 
transform is as follows: 

Break a into sub-strings by successively selecting and removing 
its "smallest" suffix. Generate the rotations of each such sub- 
string, and sort all these rotations together. The transform S is 
then obtained by taking the u last" character of this sorted list. 
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File 


Size 


BWT-compression 


5-compression 


Gain 








bytes 


ratio 


bytes 


ratio 


absolute 


relative 


BIB 


111,261 


32,022 


28.78% 


31,197 


28.04% 


0.74% 


2 


58% 


B00K1 


768,771 


242,857 


31.59% 


235,913 


30.69% 


0.90% 


2 


86% 


BOOK2 


610,856 


170,783 


27.96% 


166,881 


27.32% 


0.64% 


2 


28% 


GEO 


102,400 


66,370 


64.81% 


66,932 


65.36% 


-0.55% 


-0 


85% 


NEWS 


377,109 


135,444 


35.92% 


131,944 


34.99% 


0.93% 


2 


58% 


OBJ1 


21,504 


12,727 


59.18% 


12,640 


58.78% 


0.40% 





68% 


OBJ2 


246,814 


98,395 


39.87% 


94,565 


38.31% 


1.55% 


3 


89% 


PAPER1 


53,161 


19,816 


37.28% 


18,931 


35.61% 


1.66% 


4 


47% 


PAPER2 


82,199 


28,084 


34.17% 


27,242 


33.14% 


1.02% 


3 


00% 


PAPER3 


46,526 


18,124 


38.95% 


17,511 


37.64% 


1.32% 


3 


38% 


PAPER4 


13,286 


6,047 


45.51% 


5,920 


44.56% 


0.96% 


2 


10% 


PAPER5 


11,954 


5,815 


48.64% 


5,670 


47.43% 


1.21% 


2 


49% 


PAPER6 


38,105 


14,786 


38.80% 


14,282 


37.48% 


1.32% 


3 


41% 


PIC 


513,216 


59,131 


11.52% 


52,406 


10.21% 


1.31% 


11 


37% 


PROGC 


39,611 


15,320 


38.68% 


14,774 


37.30% 


1.38% 


3 


56% 


PROGL 


71,646 


18,101 


25.26% 


17,916 


25.01% 


0.26% 


1 


02% 


PROGP 


49,379 


13,336 


27.01% 


13,010 


26.35% 


0.66% 


2 


44% 


TRANS 


93,695 


22,864 


24.40% 


22,356 


23.86% 


0.54% 


2 


22% 


Total 


3,251,493 


980,022 


30.14% 


950,090 


29.22% 


0.92% 


3.05% 


Median 


76,923 


21,340 


36.60% 


20,644 


35.30% 


0.94% 


2 


58% 



Tab. 1: Performance of BWT-based compression <S-based compression of the 
Calgary corpus. 

Missing pieces in the above description include: the exact definition of the 
manner in which suffixes are compared, the specification of the order relation 
between rotations of different length, elaborating the meaning of the phrase 
"last" character. More importantly, we will also need to explain why this, 
seemingly arbitrary process, is reversible. 

The transform S was discovered by the second author but remained un- 
published. Unfortunately, his announcements on Internet Usenet groups such 
as comp . compression on December 2007 were received with great skepti- 
cism regarding issues including feasibility, correctness, complexity and utility. 
Here we formalize and describe the algorithm in detail, prove its correctness, 
provide a linear time and space implementation, compare to related word, 
and discuss extensions ad generalizations. 
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Outline The remainder of this article is structured as follows. Section [2gives 
some basic notations and demonstrates these in a precise definition of the 
algorithm for implementing Bur rows- Wheeler transform. Then, the reader 
is reminded of the linear time algorithm for inverting the transform. We use 
the description of these two algorithms next, in Section H] for defining the S 
transform, and in Section El the algorithm for implementing its inverse. In 
Section Owe explain why the algorithm for computing S~ l is indeed correct. 
Section [7] concludes 

2 Preliminaries I: The Burrows- Wheeler Trans- 
form 

This section serves as a reminder of the details of the Burrows- Wheeler trans- 
form. It also sets up some definitions to be used later. Let S be an ordered 
set, the alphabet of characters, and let S + be the set of finite non-empty 
strings of characters chosen from S. For a string a G S, let |a|, the length 
of a, be the number of characters in a. The order relation in £ is extended 
to a total order of strings of equal length in E + , in the usual lexicographical 
way. At this stage, we leave the comparison of strings of non-equal length 
unspecified. 

We will treat strings as arrays in the C [?] programming language, so a[0] 
shall denote the first character of a, a[l] its second character, etc. Further, 
let a[— 1] denote the last character of a, a[—2] its penultimate character, and 
more generally, for i > \a\ or i < 0, let a[i] = a[i mod \a\}. 

For strings a, /3 G £, let a(3 denote the string obtained by their concate- 
nation; we say that a (respectively >) is a prefix (respectively a suffix) of 
this concatenation. For an integer m > let a m denote the string obtained 
by concatenating a onto itself m times. 

For a G S + and an integer < m < \a\ let a(m) denote the m th rotation 
of a, that is, the string obtained by removing the first m characters. of a and 
adding these at its end. More precisely, if a = f3 r y and \a\ = m, then a(n) = 
7/3. We extend this definition for all m G Z by the equivalence a(m) = 
a(m mod |a|). For example, if a = tartar, then az(l) = a(4) = a(— 2)) = 
artarta and ct(2) = a(5) = oc(— 1) = rtarta. We have 

a{i)\j] = a[i + j] (1) 

for all a G S + and i,j G Z. 

Algorithm ^ . 1 1 describes, using these notations, the first step of the Burrows- 
Wheeler transform, that is, the generation of the list of rotations of a given 
string. 
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Algorithm 2.1 CyclicRotations(a) 


/ / Return the set of all cyclic rotations 


of a £ £+. 




1: let n = \a\ 




2: For i — 0, . . .n — 1 do 




3: R^RU {a(i)} 




4: Return i? 





Algorithm 12.11 requires 0(n) time and storage: Assuming that a is al- 
located in immutable storage, then each of a{i) can be represented by a 
triple of scalars: the address of the first character of a, the index i, and the 
length n = \a\. Henceforth, we tacitly assume this triple based representa- 
tion. 

The second step of the BWT transformation can now be described con- 
cisely as depicted by Algorithm 12.21 

Algorithm 2.2 Last(i?) / / Given a set R c £ + , return the string composed of the last character 
1 1 of each of the members of R, enumerated in lexicographical order. 
1: let n = \R\ 

2: let 7] be an uninitialized string of length n 

3: For % = 0, . . . , n — 1 do 

4: let a = min R 

5: rj[i] <r- a[—l] 

6: R^R\{a} 

7: Return i] 



Note that the algorithm is tantamount to sorting the input set R. If R 
has n elements, then this sorting can be done in 0(n\ogn) string com- 
parisons. Each such comparison may require 0(n) character comparisons, 
leading to an 0(n 2 \ogn) implementation. Yet, as Giancarlo, Restivo and 
Sciortino [?] observe, in the case that R is indeed a set of rotations, then the 
sorting can be done in 0(n) time, by reduction to the problem of sorting the 
suffixes of a given string, which is known to be linear time. 

Functions CyclicRotations (Algorithm 12. ip and Last (Algorithm 12. 2j) are 
combined in Algorithm 12. 3[ which realizes the Burrows- Wheeler transform. 
Evidently, the algorithm requires linear time and space. 
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Algorithm 2.3 BWT(a) // Given a string a € H + , return its Burrows- Wheeler 
transform. 



1: R 4— CyclicRotations(a:) 
2: Return Last(i?) 



3 Preliminaries II: Inverting the Burrows- Wheeler 
Transform 

Let a e S + be a string of length n, and let rj = BWT(a), then, examin- 
ing Algorithm I2.3I we see that it effectively defines a permutation it, such 
that rj[i] = a[ir(i)] for i — 0, 1, . . . , n — 1. Given rj, we would like to gener- 
ate the inverse permutation 7r _1 . Unfortunately, as explained above, this is 
impossible. 

Instead, the algorithm conceived by Burrows and Wheeler produces from r\ 
a permutation °d from which a rotation of a can be generated. The defining 
property of permutation $ is 



That is, having matched a position % in a with a position k in 77, we can 
match, £ — 1 mod n, the cyclically preceding location in a with the posi- 
tion d(k) in 77. Let us therefore define a permutation £> by applying d upon 
itself successively, i.e., 



Applying g to reorder the characters in r] generates, last to first, the 
characters of some rotation of a. More precisely, Burrows and Wheeler's 
inversion procedure generates the string (3, defined by f3[j] = r)[g(n — j)], 
which is not only a cyclic rotation of a, it is the lexicographically smallest 
such rotation. 

The full process of generating (3 from $ is described in Algorithm 13.11 
The algorithm is rather straightforward, except for Line[3l which initiates 
the threading process from the first character for 77. By doing that, we ensure 
that the smallest rotation of a is returned. To see that, observe that the last 
character of this smallest rotation is the one mapped by the transform to 77 [0]. 
In mapping back this character to the last character of the output, as done 
in the first time Line [5] is executed, we ensure that the we generate precisely 
this rotation. 



Vfc • (0 < k < n) A (k 



7T(i)) 



= Tr(i — 1 mod n). 



(2) 
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Algorithm 3.1 Thread^, i?) // Given the transform n = BWT(a), and the 

cyclic permutation ft, 



II return the string j3, the lexicographically smallest rotation of a. 



1 


let n = 




2 


let be an uninitialized string of length n 




3 


k^O 




4 


For i = n — 1, n — 2, . . . , do 


/ / fill in f3, last to first 


5 






6 


fc <- (fc) 




7 


Return /3 





Concentrate on the sorted list of the rotations a(i), which we will denote 
by L. The following two lemmas establish the means for generating the 
permutation •§ from r). 

Lemma 1. If for some i, < i < n, rotation a(i + 1) occurs at position k 
in L, while a(i) occurs at position j in it, then j = $(k). 

Proof. We have that 77 [k] = a(i+l)[— 1] = a[i], and^[j] = a{i)[— 1] = a{i— 1]. 
Thus, if we knew that rj[k] is mapped to a certain position in /3, we will be able 
to conclude that rj [j] is mapped to the cyclically previous position in (3. □ 

Observe that Lemma [1] does not require the knowledge of i. All we need 
to know is that the rotation at position k is obtained by omitting the first 
character of the rotation at position j, what is called by Burrows and Wheeler 
a match between j and k. 

Lemma 2. For an arbitrary character cGE, consider the sorted list a(io), ■ ■ ■ , 
of those rotations a{i), < i < n, for which a(i)[— 1] = c (that is, the ro- 
tations which correspond to occurrences of c in rj). Then, the list a(io — 
1), . . . , a(i£-i — 1) is also sorted. Moreover, this list occurs consecutively 
in L. 

Proof. Since the first character of each of the rotations a(io — 1), ... , a(^_i — 
1) is c, we can rewrite these as ca(io), • • • , ca(^_i). This list is sorted since 
we assumed that a(i ), . . . , a(^_i) are sorted. Further, the elements of this 
list occur consecutively in L since they all begin with c and no other rotation 
begins with c. □ 

Lemma [2] provides the means for matching the location of a rotation a(i) 
with the location of the rotation aii — 1). To understand the process, consider 
first the case that c is the smallest character occurring in 77, and that it is 



S 



found at locations k 0l . . . , kf_i in it. We know that there are some z , . . . , ie-i, 
such that a(io)[— 1] = a(ii)[— 1] = ••■ = 1] = c. Also, the rota- 

tions a(io), . . . , a{if^i) are sorted into into locations k , . . . , Although 
we do know the values io, ■ ■ ■ , i^-i, we can use Lemma[2]to infer the locations 
of the "preceding" rotations a(io — 1) . . . , a(i£ — 1): By this lemma, these 
must occur in L together and at the same order. Since c is the smallest 
character in 7], we can infer that these preceding rotations occur precisely at 
locations 0, 1, — 1. Matching the location in L of each a(ij) with that 
of a(ij — 1), and applying Lemma [1]^ times we conclude that 



Having done that, we can continue to the second smallest character oc- 
curring in 7], and repeat the process, except that this time, the preceding 
rotations must occur at location I in L. So, if this character is found in 
locations k' Q , . . . , we have 



Algorithm 13 . 2 1 applies this process to create The algorithm uses charac- 
ters of the alphabet E as array indices, tacitly assuming that E = {0, . . . , |E| — 1}. 

Lines [T] through |H] in the algorithm are mundane; their main purpose is 
to compute the contents of array before, which, at its c th position contains 
the number of times a character strictly smaller than c occurs in the in the 
input. 

The heart of the algorithm is in lines [TD] through O This loop effectively 
implements the process described above for each of the characters that occur 
in r\. The tricky part is that this is done simultaneously for all characters. 
Thus, instead of iterating over the different characters in 77, and then, ex- 
amining for each character all its locations, the loop in line [10] scans the 
positions in 77 in order. Array seen records at position c, the number of times 
that c was seen in course of this scan. 

Line [15] is the essence of the loop; this line generalizes ([3]) and (j3j). The 
value of before[c] provides the baseline, that is, the locations which are re- 
served for smaller characters (these locations were matched in previous itera- 
tions, or will be matched by subsequent iterations of this loop), while seen[c] 
is the number of matches of c-locations which were recorded in previous 
iterations of this loop into 

Finally, Algorithm I3.3[ combines functions Thread (Algorithm 13. ip and 
Match (Algorithm I3.2j) for inverting the Burrows- Wheeler transform. 



•&(k Q ) = o,#(k 1 ) = i,...,#(k t - 1 ) = e-i. 



(3) 



(4) 
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Algorithm 3.2 Match(?]) // Given a string n € S + ; return the permutation 



1: let n = |?7| // determine the input's length 

2: let counts be a zero initialized array of size |E| 

3: For % — 0, . . . , n — 1 do // sei counts[c] = \{i\ n[i] = c}\ for all c € £ 

4: let c = ?7 [z] // i/te character in the input we currently inspect 

5: counts[c] <— counts[c] + 1 // count this occurrence of c 

6: let before be a zero initialized array of size |E| 

7: For c = 2, . . . , |E| do //set before[c] = \{i \ r)[i] < c}\ for ailceS 

8: beforefc] = beforefc — 1] + before[c] //standard prefix sum 

9: let seen be a zero initialized array of size |E| 

10: For i — 0, . . . , n — 1 do / / set i)(i) to the next available match 

1 1 : let c = 7] [i] / j the character in the input we currently inspect 

12: «— before[c] + seen[c] / / locations 0, . . . , beforefc] — 1 are reserved 

for d < c, 

/ / while locations before[c], . . . , before[c] + seen[c] — 1 
/ / were used for earlier occurrences of c 
13: seen[c] ^— seen[c] + 1 // mark this occurrence of c as seen 

14: Return $ 



Algorithm 3.3 BWT l {r]) / / For a string rj G S + , return the smallest string (3, such that BWT(/3) = 
1: let d = Match(r/) 
2: Return Thread^, 
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4 The Bijective String Sorting Transform 



In this section, we present the iS-transform, our bijective- Burrows- Wheeler- 
like string sorting transform and its inverse S^ 1 . The outline of the algo- 
rithm for computing S is similar to that of the Burrows- Wheeler including 
computing rotations, sorting these, and then selection of the last character. 

The main difference is that the 5-transform does not work on the entire 
input as a whole. Instead, given a string a, the transform decomposes it into 
words, 

a = luqLUx • - ■ (5) 

and then proceeds to computing the rotations of each of these words, sorting 
all of the rotations together, and then selecting the last character of the 
rotations in their sorted order. The details are supplied Algorithm 14.11 

Algorithm 4.1 S(a) / / Given a string a € S + , return the bijective string 

sorting transform S(a). 

1: W <— Factor(a) // compute the Lyndon factorization of a 

2: R <— I j R will be the set of rotations of these fragments 

3: For all u> G W do / / retrieve all rotations of oj 

4: R 4— RU CyclicRotations(w) // and collect these into R 
5: Return Last(i?) 



The algorithm uses as subroutines function CyclicRotations (presented 
above in Algorithm 12. ip to produce all the cyclic rotations of the fragments, 
and function Last (presented above in Algorithm 12 .2 j) for sorting these, and 
selecting their last element. 

The factorization (jSJ) is such that each Ui is a Lyndon [?], i.e., a word 
which is smaller than all of its rotations, a;, < for j — 1, 2, . . . , \oui\ — 1. 

It is also required that < u\ < ■ ■ ■ < u m -i. The presentation of a in 
the form (j5J) ssatisfying these two properties is known as the Lyndon factor- 
ization. It is well known that the Lyndon factorization is unique. Function 
Factor called in [T] in the Algorithm 14.11 uses Duval's [?]. algorithm for com- 
puting the Lyndon factorization in linear time and space. 

Recall that we have left open the issue of extending the order relation 
in E to strings of unequal length in S + . Our transform works with two pos- 
sible such extensions, the usual lexicographical comparison in which if a is a 
prefix of (3 then a < (3. The other extension, which can be viewed as slightly 
more elegant, is that in comparing two string of unequal length, we compare 
the infinite periodic repetitions of each of these, or, phrased differently, com- 
paring a'' 3 ' with /3' a L Consider for example the strings "t/ie"' and 11 there". 
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Then, the < there according to the first definition, while the > there 
according to the infinite-periodic order since the - ■ ■ > therethere ■ ■ ■. 

Interestingly, the Lyndon factorization algorithm works for both vari- 
ations of the "lexicographical" orders. The point where these differ is in 
sorting together the the rotations of these Lyndon words. For the first, sim- 
ple and standard (but somewhat less elegant), definition we can sort the 
rotations together using the linear time suffix array construction algorithm 
of Karkkainen, Sanders and Burkhardt [?] . 

Recall that Karkkainen et. al's algorithm sorts the suffixes recursively, 
where in each recursive step, the algorithm partitions the input into char- 
acter triples, where each character triple is considered a new character. A 
linear time, radix sort, algorithm is applied to the new set of characters, 
and the recursion continues only if these new characters are not all distinct. 
In applying this algorithm to sorting the rotations of a single, non-periodic 
string (as we have in the Lyndon decompositions) u>, it is sufficient to sort 
the suffixes of uw. Thus, we can sort set of rotations of a single Lyndon 
factor separately. 

Consider now the problem of comparing rotations of of the factors luq, uui, . . . , w m _i, 
where we leave aside the issue of comparing rotations of the same Lyn- 
don factor. We deal with this specific problem using the same paradigm 
of Karkkainen et. al's, except that the grouping together of the triples is a 
cyclic fashion is done in a cyclic fashion. Of course, if the size of some Ui 
is not divisible by three, the recursive step does not reduce the number of 
characters in it. In the j th recursive step, the algorithm thus manipulates 
"characters" which belong to sequences of length 3^ in the input. To ensure 
that only linear work is done, we prune at the j th step those characters which 
belong to factors whose length is no greater than 3? . This pruning is carried 
out even if these characters are not unique! 

Merging the result of the inter-factor and intra-factor sorting steps can 
be easily done in linear time. Unfortunately, this technique only works in 
linear time for the standard lexicographical order. Sorting according to the 
infinite periodic order shall require O(nlgn) time. 

Note. The Lyndon factorization probably accounts for the better compres- 
sion results achieved by the S-transform. Recall that the standard applica- 
tion of BWT breaks the input into blocks, and then applies the "block- 
sorting" procedure to each block. In using S-transform, we break each block 
into smaller blocks, the Lyndon words, and apply a similar (but not identical) 
process to each such word. Now, the fact that this refining breakdown into 
blocks is not arbitrary, but rather depends on underlying properties of the 
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input, may very well be the reason for the better performance we witness. 

5 Inverting the String-Sorting Bijective Trans- 
form 

Algorithm 15 . 1 1 gives the procedure for inverting the transform, S^ 1 . 

Algorithm 5.1 S^ 1 ^) / / For a string 77 € S + , return the string a, such 

that S(a) = r\. 

1: let ■& = Match (77) 

2: Return MultiThread(?7, #) 



Evidently, just like the inversion of the Burrows- Wheeler transform, the 
inversion S^ 1 relies on an auxiliary permutation which plays a similar 
role in both inversions. Rather surprisingly, the same function Match (recall 
Algorithm 13.21 above) can be used for generating the permutation The 
difference is that this time 1? is not cyclic. Instead, applying Match returns 
a permutation •& which has m cycles, each corresponding to a word c^. 

More specifically, traversing the cycle 0, $(0), i?(i9(0)), . . . , 7? _1 (0) produces 
the word w m -i, last to first character. That is to say, if 77 = S(a), then the 
last character of w m -i (and hence of a) is 77 [0], the once preceding it is 77 [^(0)] , 
etc. 

Let k now be the smallest integer which is not included in this first cycle. 
Then, traversing the cycle k, ~&(k), . . . , t9 _1 (/c) produces Co> m _ 2 , again, 

from last character to first. The remaining words are produced by carrying 
out this process iteratively. 

Algorithm 15.11 thus cannot use function Thread (Algorithm 13.11 above) to 
reconstruct the original string a. Instead, it uses a more general function, 
MultiThread, for traversing the permutation Curiously, function Multi- 
Thread is a true generalization of Thread, in the sense that the call to Thread 
in the inverse Burrows- Wheeler transform, Algorithm 13.31 can be transpar- 
ently be replaced by a call to MultiThread. 

The details of MultiThread are depicted in Algorithm 15. 2\ which given a 
string 77 G cr + , 77 = Acronym (a), and a permutation with the properties as 
described above, traces the cycles in 1?, to produce the inverse transform a, 
last character to first, out of 77. 

Lines [1] through H] in this algorithm are mundane. They produce a tem- 
porary array T, which initially reflects the permutation As we traverse 
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Algorithm 5.2 MultiThread(r], i?) // Given a string rj = S(a), and the 

permutation return a. 



1 


let n = \r]\ 


// determine the input's length 


2 


let T be an uninitialized integers array of length n // used for tracking 




cycles in f} 




3 


For i — 0, . . . , n — 1 do / / initialize T with the permutation $ 


4 


T[i] <- 


j j initialize the i th position 


5 


let a be an uninitialized string of length n j j to be filled with the result, 




last to first 




6 


i ^— n — 1 


j j i is the next position in a to be filled 


7 


For j — 0, . . . , n do 


j j follow all cycles defined by ■& 


8 


If T[i]^ JL then 


j j a new cycle, starting at i, was discovered 


9 


j 


II k is used for traversing the cycle beginning at i 


10 


Repeat 


If traverse each element in the cycle which begins at i 


11 


a[i] -v- n[k] 


If produce the next output character 


12 


i -(— i — 1 


II and step back to the next output character to fill 


13 


t <- fe 


II t stores the previous value of k 


14 


fe <- T[k] 


/ / proceed to the next element in the cycle 


15 


T[t] i- JL 


/ / mark previous element as visited 


16 


until T[fc] = J. 


/ / the current cycle was exhausted 


17 


Return a 





the cycles of we mark each traversed element by setting the corresponding 
value of array T to _L. 

Next, the algorithm proceeds to producing the returned string a, starting 
at its last character, working its way to its first. The outer loop (lines 0- 
[T6|) examines each position in array T in turn. If the value stored in this 
position is not _L, then the inner loop (lines [T01TT6]) follows up the cycle in i? 
that starts at this position, producing an output character in each iteration, 
and marking each visited position by setting the corresponding location of T 
to _L 

6 Correctness of the Inversion Algorithm 

It is easy to check that algorithms Algorithm 15.11 and Algorithm 15.21 require 
linear time and space. We now turn to the issue of their correctness. 

Examining Algorithm I4.1[ we see that it effectively computes a permuta- 
tion 7r of the input. A position i in the input string a is first associated with 
a certain word u £ W. Exactly one of the rotations of u is such that this 
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position comes to be the last character. The sorting together of all rotations 
of the words in W assigns an ordinal number to this rotation; this ordinal 
number is nothing else than 7r(z). 

Given 77 = S(a), finding the inverse of 7r is done again by computing the 
auxiliary permutation 1?, but this time, $ is defined in a piecemeal fashion. 
Let ti u : {0, . . . , \u\ — 1} — > {0, . . . , n — 1} be the function describing the 
mapping from the positions of a word u into positions of 77 as carried 
out by S. The defining property of is 

Vfc, (0 < k < n) A (k = tt^O')) 0(jfe) = 7r(i - 1 mod |w|). (6) 

% That is, having matched a position in 77 not only with some word u but 
also with a position i in that word, we can match position i — 1 mod \u\, the 
cyclically preceding position, in u, with 

To understand why Algorithm 13.21 computes $ also for the 5 transform, 
let us consider the general setting in which we sort together rotations of 
multiple words. Henceforth, let W C S + be a fixed finite set of words, 
and let n = XLew M> be ^ ne total length of all the words in W. Also, 
let L = LqL\ ■ ■ ■ L n _\ be the sorted list of all rotations of the words in W, 
so each Lj is a rotation of some word in W, and let rj be the string defined 
by V [k] = L k [-l}. 

The following generalizes Lemma [TJ 

Lemma 3. If Lj = L k (—1) then j = 

Proof. The proof is essentially the same as that of the proof of Lemma [TJ 
From the assumptions it follows that there is a word u G W and an index % 
such that Lfc = uj(i + 1) and Lj = u;(z). The last character in is there- 
fore uj[i] while the last character in Lj is u[i — 1]. Thus, if we knew that 
is mapped to a certain position inw, we will be able to conclude that rj[j] is 
mapped to the cyclically previous position in to. □ 

Lemma 4. For an arbitrary c£ E, let L io , . . . , L il _ 1 be the list of all rota- 
tions L in which c occurs as the last character, that is r][i ] = 77 [«i] = ■ ■ ■ = 
77^-1] = c. Then, the list L io (— 1), . . . , L^_ 1 (— 1) occurs consecutively and in 
that order in L. 

Proof. Since the first character of each of the rotations Lj (— 1), . . . , L^_ 1 (— 1) 
is c, we can rewrite these as cLj , . . . , cL^ x . This list is sorted since we 
assumed that a()io, . . . , r 1 ^ 1 are sorted. Further, the elements of this list 
occur consecutively in L since they all begin with c and no other rotation 
begins with c. □ 
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7 Future Work 

Clearly, the work ahead of us is in evaluating the efficacy of the transform 
described here in state of the art compression programs such as bzip2 and 
7-zip. 

We are intrigued by the question of sorting the rotations of the Lyndon 
decomposition in linear time with the infinite periodic order. 
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